Vehicle surrounding area image generation device, vehicle surrounding area display system, and vehicle surrounding area display method

ABSTRACT

An ECU generates an overhead image including a vehicle and a vehicle surrounding area, on the basis of a plurality of images taken by a plurality of cameras mounted on the vehicle. When a distance between an object existing in an overlapping portion of imaging areas of the plurality of cameras and the vehicle is within a threshold and the object exists in a traveling direction of the vehicle, the overlapping portion is divided into two pairs of a road surface image and an object image, and combination is performed by alpha blending with an α value for the object image being set to 1 and an α value for the road surface image being set to 0 for an area that is the object image in one of the two pairs and is the road surface image in the other of the two pairs, to generate the overhead image.

TECHNICAL FIELD

The present disclosure relates to a vehicle surrounding area imagegeneration device, a vehicle surrounding area display system, and avehicle surrounding area display method.

BACKGROUND ART

In a vehicle surrounding area display system including a vehiclesurrounding area image generation device, images of a vehiclesurrounding area taken by a plurality of on-vehicle cameras are combinedto generate an overhead image including the vehicle and the vehiclesurrounding area. The driver recognizes a person, an object, or the likeexisting around the vehicle through the overhead image displayed on adisplay device. For example, Patent Document 1 indicates that pixelinformation of each input image is alternately adopted according to acertain rule for the same imaging area included in a plurality of inputimages, and a composite image is outputted.

When a plurality of images are combined to generate an overhead image,the images overlap at the boundary portion between the imaging areas oftwo cameras. In the conventional combining method, a combining ratio ofeach image (hereinafter, referred to as a value) is set to 0<α<1 forthis overlapping portion. The technique of combining two images with acertain α value as described above is called alpha blending. In alphablending, for example, when the α value is set to 0.5, a color that is amixture of 50% of the colors of the pixels of the overlapping portion ineach of two images is generated, and an average image is synthesized.

CITATION LIST Patent Document

-   Patent Document 1: Japanese Laid-Open Patent Publication No.    2003-169323

SUMMARY OF THE INVENTION Problems to be Solved by the Invention

When an overhead image is synthesized in the conventional vehiclesurrounding area image generation device, two images are combined byalpha blending with the overlapping portion in each of the two imagesbeing made transparent. In this case, the α value is set regardless ofwhether an object exists in the image. Therefore, there is a problemthat a part of an object existing in the overlapping portion of theimaging area of each camera is blurred in the overhead image and thusthe overhead image becomes an unclear image. Moreover, in alphablending, even if the overlapping portion is weighted in proportion tothe distance from the center of the imaging area, an object may beblurred.

The present disclosure has been made to solve the above problem, and anobject of the present disclosure is to obtain a vehicle surrounding areaimage generation device capable of generating a clear overhead imagewithout blurring an object existing in an overlapping portion of theimaging areas of a plurality of imaging devices. In addition, anotherobject of the present disclosure is to obtain a vehicle surrounding areadisplay system and a vehicle surrounding area display method capable ofclearly displaying an object existing in the overlapping portion of theimaging areas.

Solution to the Problems

A vehicle surrounding area image generation device according to thepresent disclosure is a vehicle surrounding area image generation devicefor generating an overhead image including a vehicle and a vehiclesurrounding area, on the basis of a plurality of images taken by aplurality of imaging devices mounted on the vehicle, the vehiclesurrounding area image generation device including: an object distancedetermination unit for determining whether a distance between an objectand the vehicle is within a threshold, on the basis of objectinformation acquired by a sensor for detecting the object in the vehiclesurrounding area; an image conversion unit for converting each of theplurality of images into an image for an overhead image; an imagedivision unit for, when the object exists in an overlapping portion ofimaging areas of the plurality of imaging devices, dividing theoverlapping portion in the image for an overhead image into a roadsurface image and an object image; and an image combining unit forcombining all or part of the image for an overhead image obtained by theimage conversion unit and the road surface image and the object imageobtained by the image division unit respectively, to generate theoverhead image.

A vehicle surrounding area display system according to the presentdisclosure is a vehicle surrounding area display system including: aplurality of imaging devices, mounted on a vehicle, for taking an imageof a vehicle surrounding area; a sensor for detecting an object in thevehicle surrounding area; a vehicle surrounding area image generationdevice for generating an overhead image including the vehicle and thevehicle surrounding area, on the basis of a plurality of images taken bythe imaging devices; and a display unit for displaying the overheadimage generated by the vehicle surrounding area image generation device,wherein the vehicle surrounding area image generation device includes anobject distance determination unit for determining whether a distancebetween the object and the vehicle is within a threshold, on the basisof object information acquired by the sensor, an image conversion unitfor converting each of the plurality of images into an image for anoverhead image, an image division unit for, when the object exists in anoverlapping portion of imaging areas of the plurality of imagingdevices, dividing the overlapping portion in the image for an overheadimage into a road surface image and an object image, and an imagecombining unit for combining all or part of the image for an overheadimage obtained by the image conversion unit and the road surface imageand the object image obtained by the image division unit respectively,to generate the overhead image.

A vehicle surrounding area display method according to the presentdisclosure is a vehicle surrounding area display method for generatingand displaying an overhead image including a vehicle and a vehiclesurrounding area, the vehicle surrounding area display method including:an information acquisition step of acquiring image information of aplurality of taken images of the vehicle surrounding area, objectinformation including position information of an object in the vehiclesurrounding area, and vehicle traveling information including atraveling direction and a speed of the vehicle; an image conversion stepof converting each of the plurality of images acquired in theinformation acquisition step, into an image for an overhead image; aroad surface color detection step of, when the object exists in anoverlapping portion of the plurality of images and a distance betweenthe object and the vehicle is within a threshold, detecting a roadsurface color of the overlapping portion; an image division step ofdividing the overlapping portion in the image for an overhead image intoa road surface image and an object image on the basis of the roadsurface color detected in the road surface color detection step; animage combining step of combining all or part of the image for anoverhead image obtained in the image conversion step and the roadsurface image and the object image obtained in the image division steprespectively, to generate the overhead image; and a display step ofdisplaying the overhead image generated in the image combining step.

Effect of the Invention

In the vehicle surrounding area image generation device according to thepresent disclosure, when an object exists in the overlapping portion ofthe imaging areas of a plurality of cameras, the overlapping portion isdivided into a road surface image and an object image, and an image foran overhead image, the road surface image, and the object image arecombined to generate an overhead image. Thus, it is possible to generatea clear overhead image without blurring the object existing in theoverlapping portion.

In the vehicle surrounding area display system according to the presentdisclosure, when an object exists in the overlapping portion of theimaging areas of a plurality of cameras, the overlapping portion isdivided into a road surface image and an object image, and an image foran overhead image, the road surface image, and the object image arecombined to generate an overhead image. Thus, it is possible to displaya clear overhead image without blurring the object existing in theoverlapping portion.

In the vehicle surrounding area display method according to the presentdisclosure, when an object exists in the overlapping portion of theimaging areas of the plurality of cameras and the distance between theobject and the vehicle is within the threshold, the overlapping portionis divided into a road surface image and an object image, and an imagefor an overhead image, the road surface image, and the object image arecombined to generate an overhead image. Thus, it is possible to displaya clear overhead image without blurring the object existing in theoverlapping portion.

Objects, features, aspects, and effects of the present disclosure otherthan the above will become more apparent from the following detaileddescription with reference to the drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram showing the configuration of a vehiclesurrounding area display system according to Embodiment 1.

FIG. 2 is a block diagram showing the configuration of a vehiclesurrounding area image generation device according to Embodiment 1.

FIG. 3 is a diagram showing a hardware configuration example of thevehicle surrounding area image generation device according to Embodiment1.

FIG. 4 is a diagram illustrating an image combining method by thevehicle surrounding area image generation device according to Embodiment1.

FIG. 5 is a diagram illustrating the image combining method by thevehicle surrounding area image generation device according to Embodiment1.

FIG. 6 is a diagram showing an example of an overhead image displayed ona display unit of the vehicle surrounding area display system accordingto Embodiment 1.

FIG. 7 is a diagram showing an example of an overhead image displayed ona display unit of a vehicle surrounding area display system according toa comparative example.

FIG. 8 is a diagram illustrating flow of processing by the vehiclesurrounding area display system according to Embodiment 1.

FIG. 9 is a diagram illustrating a method for selecting an imagecombining method by a vehicle surrounding area image generation deviceaccording to Embodiment 2.

FIG. 10 is a diagram illustrating the method for selecting an imagecombining method by the vehicle surrounding area image generation deviceaccording to Embodiment 2.

FIG. 11 is a diagram illustrating flow of processing by a vehiclesurrounding area display system according to Embodiment 2.

DESCRIPTION OF EMBODIMENTS Embodiment 1

Hereinafter, a vehicle surrounding area image generation device and avehicle surrounding area display system according to Embodiment 1 willbe described with reference to the drawings. FIG. 1 is a block diagramshowing the configuration of the vehicle surrounding area display systemaccording to Embodiment 1, FIG. 2 is a block diagram showing theconfiguration of the vehicle surrounding area image generation deviceaccording to Embodiment 1, FIG. 3 is a diagram showing a hardwareconfiguration example of the vehicle surrounding area image generationdevice, and FIG. 4 and FIG. 5 are diagrams illustrating an imagecombining method by the vehicle surrounding area image generationdevice. In the drawings, identical and corresponding portions aredesignated by the same reference characters.

As shown in FIG. 1, the vehicle surrounding area display system includesa plurality of cameras 1 which are imaging devices for taking an imageof a vehicle surrounding area, an ECU 2 (electronic control unit) whichis the vehicle surrounding area image generation device, a vehicletraveling information detection unit 3, a sensor 4 for detecting anobject around a vehicle, an object information detection unit 5, and adisplay unit 6.

As shown in FIG. 4, for example, the cameras 1 are attached to thefront, rear, left, and right of a vehicle 7, and take images of thefront, rear, left, and right areas of the vehicle 7. The imaging areasby the respective cameras 1 have overlapping portions. In the exampleshown in FIG. 4, an imaging area 10B by the camera 1 in the rear of thevehicle and an imaging area 10R by the camera 1 on the right side of thevehicle have an overlapping portion 10BR in a right rear area of thevehicle 7. In the case of including four cameras 1, there are fouroverlapping portions.

The ECU 2 generates an overhead image including the vehicle 7 and thevehicle surrounding area on the basis of a plurality of images of thevehicle surrounding area taken by the cameras 1. The functions of theECU 2 will be described in detail later. The vehicle travelinginformation detection unit 3 acquires vehicle traveling informationincluding the traveling direction and the speed of the vehicle 7, fromthe vehicle 7. Specifically, the vehicle traveling information detectionunit 3 acquires vehicle information such as the current steering angle,speed, and gear information, from an engine control unit, and detectsthe traveling direction and the speed of the vehicle 7 on the basis ofthe vehicle information.

The sensor 4 is attached to the vehicle 7, detects an object 8 in thevehicle surrounding area, and acquires object information. For example,as shown in FIG. 4, sensors 4 are each provided between a camera 1 and acamera 1, and are each set such that a detection range by the sensor 4includes the overlapping portion of the imaging areas of the two cameras1. The type of each sensor 4 is not particularly limited, and amillimeter wave radar, a sonar which is an ultrasonic sensor, a camera,or the like is used.

The object information detection unit 5 acquires the object informationincluding position information of the object 8 detected by the sensor 4,and detects the distance between the object 8 and the vehicle 7 on thebasis of the position information. A sensor having the function of theobject information detection unit 5 may be used. The display unit 6displays the overhead image that is generated by the ECU 2 and thatincludes the vehicle 7 and the vehicle surrounding area. The displayunit 6 is a general display and is installed in the vehicle interiorsuch that the driver can easily see the display unit 6.

As shown in FIG. 2, the ECU 2 includes a frame synchronization unit 21,an object distance determination unit 22, a combining method selectionunit 23, an image conversion unit 24, a road surface color detectionunit 25, an image division unit 26, and an image combining unit 27.Hereinafter, the functions of the respective units will be described.

The frame synchronization unit 21 synchronizes image informationacquired from the cameras 1, the object information acquired from theobject information detection unit 5, and the vehicle travelinginformation including the traveling direction and the speed of thevehicle 7. The vehicle traveling information is detected by the vehicletraveling information detection unit 3. Frame synchronization is one ofthe signal synchronization methods, and, in the frame synchronization,synchronization is performed according to a predetermined framesynchronization pattern. The object distance determination unit 22, thecombining method selection unit 23, and the image conversion unit 24each perform processing using the information synchronized by the framesynchronization unit 21.

The object distance determination unit 22 determines whether thedistance between the object 8 and the vehicle 7 is within a threshold,on the basis of the object information acquired from the objectinformation detection unit 5. Whether the object 8 is displayed in theoverhead image displayed on the display unit 6 depends on the positionof the object 8 and the distance between the object 8 and the vehicle 7.Therefore, the threshold used in the object distance determination unit22 is set on the basis of the distance from an end portion of thevehicle 7 in the overhead image shown in FIG. 4 to a screen edge 6A(indicated by an alternate long and short dash line in FIG. 4) of thedisplay unit 6.

The combining method selection unit 23 selects a combining method for anoverhead image in the image combining unit 27 on the basis of theframe-synchronized image information, object information, and vehicletraveling information. In Embodiment 1, when the object 8 exists in theoverlapping portion of the imaging areas of two cameras 1, and thedistance between the object 8 and the vehicle 7 is within the threshold,the combining method selection unit 23 selects a first combining method.On the other hand, when no object 8 exists in the overlapping portion orwhen the distance between the object 8 and the vehicle 7 is larger thanthe threshold, the combining method selection unit 23 selects a secondcombining method. The first combining method and the second combiningmethod will be described in detail later.

The image conversion unit 24 acquires image information of a pluralityof images taken by the cameras 1, from the frame synchronization unit21, and converts each image into an image for an overhead image. Whenthe combining method selection unit 23 selects the first combiningmethod, the images for an overhead image obtained by the imageconversion unit 24 are sent to the road surface color detection unit 25and the image combining unit 27. In addition, when the combining methodselection unit 23 selects the second combining method, the images for anoverhead image obtained by the image conversion unit 24 are sent only tothe image combining unit 27.

The road surface color detection unit 25 detects the road surface colorof the overlapping portion in each image for an overhead image generatedby the image conversion unit 24. Specifically, when the combining methodselection unit 23 selects the first combining method, the road surfacecolor detection unit 25 cuts out the overlapping portion from each ofthe two images for an overhead image that have the overlapping portion,and detects the road surface color of the cut image.

As a method for detecting a road surface color, there is a method inwhich color information of an image portion (for example, several pixelsaround the vehicle 7) in which a road surface is seen is detected fromthe image information acquired by the cameras 1, and this colorinformation is used as color information of a road surface color. Also,there is a method in which position information of an object is acquiredfrom the object information detection unit 5, color information of aportion where no object exists is acquired on the basis of the positioninformation, and this color information is used as color information ofa road surface color.

When the object 8 exists in the overlapping portion of the imaging areasof multiple cameras 1, the image division unit 26 divides theoverlapping portion in each image for an overhead image into a roadsurface image and an object image. When the combining method selectionunit 23 selects the first combining method, the image division unit 26divides the overlapping portion in each image for an overhead image intoa road surface image and an object image on the basis of the colorinformation of the road surface color detected by the road surface colordetection unit 25. That is, in the image division unit 26, two pairs ofa road surface image and an object image are obtained for oneoverlapping portion.

In a method for dividing an image by the image division unit 26, forexample, the image division unit 26 searches for pixels that match thecolor information of the road surface color, from the overlappingportion in the image for an overhead image, and distinguishes between anobject portion and a road surface portion. The image division unit 26cuts out the pixels of the object portion as an object image, and fillsthis portion with the pixels that match the color information of theroad surface color, thereby generating a road surface image.

In Embodiment 1, the image division unit 26 divides the overlappingportion in the image into a road surface image and an object image onthe basis of the color information of the road surface color, but themethod for dividing an image is not limited thereto. For example, asanother method, a method, in which the coordinates of the object portionin the overlapping portion are calculated on the basis of the positioninformation of the object 8 acquired by the object information detectionunit 5 and the overlapping portion is divided into a road surface imageand an object image, may be used.

The image combining unit 27 combine all or part of the image for anoverhead image obtained by the image conversion unit 24 and the roadsurface image and the object image respectively obtained by the imagedivision unit 26, to generate an overhead image. The image combiningunit 27 generates an overhead image by either the first combining methodor the second combining method.

In the first combining method, the images for an overhead image obtainedby the image conversion unit 24 and the road surface image and theobject image respectively obtained by the image division unit 26 arecombined to generate an overhead image. When the first combining methodis selected, since the object 8 exists in the overlapping portion of theimaging areas, two pairs of a road surface image and an object imageobtained by the image division unit 26 are combined for the overlappingportion, and the images for an overhead image obtained by the imageconversion unit 24 are combined for the other portion. In addition, inthe second method, only the images for an overhead image obtained by theimage conversion unit 24 are combined to generate an overhead image.

The ECU 2 is an electronic circuit including a microcomputer and thelike, and is composed of a processor 100 and a storage device 101 asshown in FIG. 3. The storage device 101 includes a volatile storagedevice such as a random access memory, and a non-volatile auxiliarystorage device such as a flash memory. Moreover, the storage device 101may include an auxiliary storage device that is a hard disk, instead ofthe flash memory.

The processor 100 executes a program inputted from the storage device101. In this case, the program is inputted from the auxiliary storagedevice via the volatile storage device to the processor 100. Inaddition, the processor 100 may output data such as a calculation resultto the volatile storage device of the storage device 101, or may storethe data in the auxiliary storage device via the volatile storagedevice.

Moreover, the vehicle traveling information detection unit 3 and theobject information detection unit 5 are similarly composed of aprocessor and a storage device installed in the vehicle surrounding areadisplay system (not shown). A program that describes the functions ofthe vehicle traveling information detection unit 3 and the objectinformation detection unit 5 is executed by the processor.

Next, the combining method for an overhead image by the ECU 2 will bedescribed in detail with reference to FIG. 4 and FIG. 5. In FIG. 4, theobject 8 exists in the overlapping portion 10BR of the imaging area 10Bby the camera 1 in the rear of the vehicle and the imaging area 10R bythe camera 1 on the right side of the vehicle. In addition, the object 8is located inside the screen edge 6A of the display unit 6, and thus theobject distance determination unit 22 determines that the distancebetween the object 8 and the vehicle 7 is within the threshold.Therefore, the combining method selection unit 23 selects the firstcombining method.

When the first combining method is selected, the image conversion unit24 cuts out the image of a section 11 in which the object 8 is detected,from the image taken by each camera 1, and converts the cut image intoan image for an overhead image. Alternatively, the image conversion unit24 may convert the image taken by each camera 1 into an image for anoverhead image and then cut out the image of the section 11 in which theobject 8 is detected. As a result, as shown in FIG. 5, an overhead image11 a of the section 11 taken by the camera 1 in the rear of the vehicleand an overhead image 11 b of the section 11 taken by the camera 1 onthe right side of the vehicle are obtained. These overhead images 11 aand 11 b each include the object 8 and a road surface 12.

Subsequently, the road surface color detection unit 25 detects colorinformation of the road surface color in each of the overhead images 11a and 11 b. The image division unit 26 extracts the object 8 from eachof the overhead images 11 a and 11 b on the basis of the road surfacecolor detected by the road surface color detection unit 25, and divideseach of the overhead images 11 a and 11 b into a road surface imageincluding the road surface 12 and an object image including the object8. The image combining unit 27 performs image combination by alphablending using these two pairs of the road surface image and the objectimage.

In the first combining method, as shown in FIG. 5, combination isperformed with each of the overhead images 11 a and 11 b being dividedinto four areas. The first area is an area that is the road surfaceimage in both the overhead images 11 a and 11 b. For the first area,each α value is set to 0.5. The second area is an area that is theobject image in the overhead image 11 a and is the road surface image inthe overhead image 11 b. For the second area, an α value for the objectimage is set to 1, and an α value for the road surface image is set to0.

The third area is an area that is the road surface image in the overheadimage 11 a and is the object image in the overhead image 11 b. For thethird area, an α value for the object image is set to 1, and an α valuefor the road surface image is set to 0. The fourth area is an area thatis the object image in both the overhead images 11 a and 11 b. For thefourth area, each α value is set to 0.5. For the first area and thefourth area, setting of the α value is not limited thereto.

As described above, in the first combining method, when the road surfaceimage and the object image of the overlapping portion are combined, forthe area that is the object image in one of the two pairs and is theroad surface image in the other of the two pairs, the α value for theobject image is set to 1, and the α value for the road surface image isset to 0. The road surface images and the object images of the fourareas for each of which the α value is provided are combined to generatean overhead image 11 c shown in FIG. 5. The following equation (1) is ageneral equation of alpha blending using an α value. In equation (1), Cgis color information of a road surface image, Co is color information ofan object image, A is an α value, and Cr is color information aftercombination.

Cr=Cg×(1−A)+Co×A  (1)

In general alpha blending, the α value that is A in equation (1) is setbetween 0 and 1, and the colors of both images are mixed with the colorof each image being made transparent. For example, when the α value isset to 0.5, an image in which the colors of both images are mixed to thesame extent is generated. On the other hand, in the ECU 2 according toEmbodiment 1, the α value for the object image is set to 1 for thesecond area and the third area shown in FIG. 5, and combination isperformed without making the colors of the object images transparent.

FIG. 6 shows an example of an overhead image displayed on the displayunit of the vehicle surrounding area display system according toEmbodiment 1. An overhead image 13 a is obtained by dividing the imagesof the overlapping portion into two pairs of a road surface image and anobject image and performing combination with the α value for the objectimage being set to 1 and the α value for the road surface image beingset to 0 for the area that is the object image in one of the two pairsand is the road surface image in the other of the two pairs. In theoverhead image 13 a displayed on the display unit 6, rod-shaped objects9 are clearly seen in front of the vehicle 7 without blurring.

Meanwhile, FIG. 7 shows an example of an overhead image displayed on adisplay unit of a vehicle surrounding area display system according to acomparative example. In the comparative example, images for an overheadimage by two cameras 1 are combined with each α value being set to 0.5without dividing each image of an overlapping portion into a roadsurface image and an object image. Thus, the colors of object portionsof two images for an overhead image are each made transparent by 50%,and rod-shaped objects 9 seen in an overhead image 13 b are blurred.

Next, flow of processing by the vehicle surrounding area display systemaccording to Embodiment 1 will be described with reference to aflowchart of FIG. 8. First, in step S1, image information of a pluralityof taken images of the vehicle surrounding area, object informationincluding position information of an object in the vehicle surroundingarea, and vehicle traveling information including the travelingdirection and the speed of the vehicle are acquired (informationacquisition step). Subsequently, in step S2, the information acquired instep S1 is frame-synchronized (frame synchronization step).

In step S3, a combining method for an overhead image is selected on thebasis of the information obtained in step S1 and frame-synchronized instep S2 (combining method selection step). First, in step S3-1, whetheran object exists in the overlapping portion of the imaging areas in theplurality of taken images of the vehicle surrounding area is determined.When an object exists in the overlapping portion (YES), the distancebetween the vehicle and the object is acquired in step S3-2. Next, instep S3-3, whether the distance between the object and the vehicle iswithin the threshold is determined. When the distance between the objectand the vehicle is within the threshold (YES), the first combiningmethod is selected in step S3-5.

On the other hand, when no object exists in the overlapping portion instep S3-1 (NO), or when the distance between the object and the vehicleis larger than the threshold in step S3-3 (NO), the second combiningmethod is selected in step S3-6.

When the first combining method is selected in step S3-5, each of theplurality of taken images is converted into an image for an overheadimage in step S4 (image conversion step). Next, in step S5, the roadsurface color of the overlapping portion in each image for an overheadimage is detected (road surface color detection step). Furthermore, instep S6, the overlapping portion in each image for an overhead image isdivided into a road surface image and an object image on the basis ofthe road surface color detected in the road surface color detection step(S5) (image division step).

Subsequently, in step S7, using the images of the overlapping portionobtained in the image division step (S6), that is, two pairs of theobject image and the road surface image, the images are combined with anα value for the object image being set to 1 and an α value for the roadsurface image being set to 0 for an area that is the object image in oneof the two pairs and is the road surface image in the other of the twopairs, to generate an overhead image of the overlapping portion. Next,in step S8, the images for an overhead image of the other portionobtained in the image conversion step (S4) are combined to generate anoverhead image including the vehicle and the vehicle surrounding area(image combining step).

On the other hand, when the second combining method is selected in stepS3-6, each of the plurality of taken images is converted into an imagefor an overhead image in step S9 (image conversion step). Subsequently,in step S10, the images for an overhead image obtained in the imageconversion step (S9) are combined to generate an overhead imageincluding the vehicle and the vehicle surrounding area (image combiningstep). Finally, in step S11, the overhead image generated in the imagecombining step (S7, S8, S10) is displayed (display step).

As described above, in the ECU 2 according to Embodiment 1, when theobject 8 exists in the overlapping portion of the imaging areas ofmultiple cameras 1, the overlapping portion is divided into a roadsurface image and an object image, and combination is performed by alphablending with an α value for the object image being set to 1 and an αvalue for the road surface image being set to 0, to generate an overheadimage. Thus, it is possible to generate a clear overhead image withoutblurring the object 8 existing in the overlapping portion. In addition,with the vehicle surrounding area display system including the ECU 2according to Embodiment 1, it is possible to display a clear overheadimage without blurring the object 8 existing in the overlapping portion.

Moreover, in the vehicle surrounding area display method according toEmbodiment 1, when the object 8 exists in the overlapping portion of aplurality of images and the distance between the object 8 and thevehicle 7 is within the threshold, the road surface color of theoverlapping portion of each image for an overhead image is detected, theoverlapping portion is divided into a road surface image and an objectimage on the basis of the road surface color, and the image for anoverhead image, the road surface image, and the object image arecombined to generate an overhead image. Thus, it is possible to displaya clear overhead image without blurring the object 8 existing in theoverlapping portion.

Embodiment 2

The configurations of a vehicle surrounding area image generation deviceand a vehicle surrounding area display system according to Embodiment 2are the same as in Embodiment 1 described above. Thus, FIG. 1 and FIG. 2are used again, and only the differences will be described here. FIG. 9and FIG. 10 are diagrams illustrating a method for selecting an imagecombining method by the vehicle surrounding area image generation deviceaccording to Embodiment 2. In FIG. 9 and FIG. 10, an arrow A indicatesthe traveling direction of the vehicle 7.

In FIG. 9, an object 8 exists in a section 11 of an overlapping portion10FR of an imaging area 10F by the camera 1 in the front of the vehicleand the imaging area 10R by the camera 1 on the right side of thevehicle. In addition, in FIG. 10, an object 8 exists in the section 11of the overlapping portion 10BR of the imaging area 10B by the camera 1in the rear of the vehicle and the imaging area 10R by the camera 1 onthe right side of the vehicle.

In any of these cases, the object 8 is located inside the screen edge 6Aof the display unit 6, and the distance between the object 8 and thevehicle is within the threshold, but the relationship between theposition of the object 8 and the traveling direction of the vehicle 7are different between these cases. That is, in FIG. 9, the object 8exists on the traveling direction side of the vehicle 7, so that thevehicle 7 is highly likely to come into contact with the object 8. Onthe other hand, in FIG. 10, the object 8 exists on the opposite sidefrom the traveling direction of the vehicle 7, so that the vehicle 7 isunlikely to come into contact with the object 8.

When the object 8 exists in the overlapping portion, the distancebetween the object 8 and the vehicle 7 is within the threshold, and theobject 8 exists in the traveling direction of the vehicle 7, thecombining method selection unit 23 according to Embodiment 2 selects thefirst combining method in which an image for an overhead image, a roadsurface image, and an object image are used. On the other hand, when noobject 8 exists in the overlapping portion, when the distance betweenthe object 8 and the vehicle 7 is larger than the threshold, or when noobject 8 exists in the traveling direction of the vehicle 7, thecombining method selection unit 23 selects the second combining methodin which an image for an overhead image is used.

In Embodiment 1 described above, even when the object 8 exists on theopposite side from the traveling direction of the vehicle 7 as in FIG.10, the first combining method is selected, so that the calculation loadof the ECU 2 is increased. In Embodiment 2, when the object 8 exists onthe opposite side from the traveling direction of the vehicle 7, thesecond combining method is selected, so that the object 8 is blurred ina generated overhead image. However, the object 8 that exists on theopposite side from the traveling direction is less likely to come intocontact with the vehicle 7, and thus does not need to be so clear.

FIG. 11 is a flowchart illustrating flow of processing by the vehiclesurrounding area display system according to Embodiment 2. Thedifference between FIG. 11 and the flowchart of FIG. 8 described abovein Embodiment 1 is only that step S3-4 is added to the combining methodselection step (S3). That is, steps S1, S2, and S4 to S11 are the sameas in the flowchart of FIG. 8, and thus the description thereof isomitted.

In the combining method selection step (S3) according to Embodiment 2,when an object exists in the overlapping portion of the imaging areas instep S3-1 (YES), the distance between the vehicle and the object isacquired in step S3-2. Next, when the distance between the object andthe vehicle is within the threshold in step S3-3 (YES), whether theobject exists in the traveling direction of the vehicle is determined instep S3-4. When the object exists in the traveling direction of thevehicle (YES), the first combining method is selected in step S3-5.

On the other hand, when no object exists in the overlapping portion instep S3-1 (NO), when the distance between the object and the vehicle islarger than the threshold in step S3-3 (NO), or when the object does notexist in the traveling direction of the vehicle in step S3-4 (NO), thesecond combining method is selected in step S3-6.

In Embodiment 2, in addition to the same effects as those of Embodiment1 described above, the calculation load of the ECU 2 is reduced sincethe first combining method is selected only when it is determined thatthe object 8 exists in the traveling direction of the vehicle 7.

Although the present disclosure is described above in terms of variousexemplary embodiments, it should be understood that the variousfeatures, aspects, and functionality described in one or more of theindividual embodiments are not limited in their applicability to theparticular embodiment with which they are described, but instead can beapplied, alone or in various combinations to one or more of theembodiments of the disclosure. It is therefore understood that numerousmodifications which have not been exemplified can be devised withoutdeparting from the scope of the present disclosure. For example, atleast one of the constituent components may be modified, added, oreliminated. At least one of the constituent components mentioned in atleast one of the preferred embodiments may be selected and combined withthe constituent components mentioned in another preferred embodiment.

DESCRIPTION OF THE REFERENCE CHARACTERS

-   -   1 camera    -   2 ECU    -   3 vehicle traveling information detection unit    -   4 sensor    -   5 object information detection unit    -   6 display unit    -   6A screen edge    -   7 vehicle    -   8, 9 object    -   10B, 10F, 10R imaging area    -   10BR, 10FR overlapping portion    -   11 section    -   11 a, 11 b, 11 c, 13 a, 13 b overhead image    -   12 road surface    -   21 frame synchronization unit    -   22 object distance determination unit    -   23 combining method selection unit    -   24 image conversion unit    -   25 road surface color detection unit    -   26 image division unit    -   27 image combining unit

1. A vehicle surrounding area image generation device for generating anoverhead image including a vehicle and a vehicle surrounding area, onthe basis of a plurality of images taken by a plurality of imagingdevices mounted on the vehicle, the vehicle surrounding area imagegeneration device comprising: an object distance determining circuitryto determine whether a distance between an object and the vehicle iswithin a threshold, on the basis of object information acquired by asensor for detecting the object in the vehicle surrounding area; animage converting circuitry to convert each of the plurality of imagesinto an image for an overhead image; an image dividing circuitry todivide the overlapping portion in the image for an overhead image into aroad surface image and an object image when the object exists in anoverlapping portion of imaging areas of the plurality of imagingdevices; and an image combining circuitry to combine all or part of theimage for an overhead image obtained by the image converting circuitryand the road surface image and the object image obtained by the imagedividing circuitry respectively, to generate the overhead image.
 2. Thevehicle surrounding area image generation device according to claim 1,further comprising a frame synchronizing circuitry to synchronize imageinformation acquired from the imaging devices, object informationacquired from the sensor, and vehicle traveling information including atraveling direction and a speed of the vehicle.
 3. The vehiclesurrounding area image generation device according to claim 2, furthercomprising a combining method selecting circuitry to select a combiningmethod for the overhead image in the image combining circuitry on thebasis of the image information, the object information, and the vehicletraveling information synchronized by the frame synchronizing circuitry.4. The vehicle surrounding area image generation device according toclaim 3, wherein when the object exists in the overlapping portion andthe distance between the object and the vehicle is within the threshold,the combining method selecting circuitry selects a first combiningmethod in which the image for an overhead image, the road surface image,and the object image are used, and when the object does not exist in theoverlapping portion or when the distance between the object and thevehicle is larger than the threshold, the combining method selectingcircuitry selects a second combining method in which the image for anoverhead image is used.
 5. The vehicle surrounding area image generationdevice according to claim 4, wherein when the object exists in thetraveling direction of the vehicle, the combining method selectingcircuitry selects the first combining method, and when the object doesnot exist in the traveling direction of the vehicle, the combiningmethod selecting circuitry selects the second combining method.
 6. Thevehicle surrounding area image generation device according to claim 1,further comprising a road surface color detector to detect a roadsurface color of the overlapping portion, wherein the image dividingcircuitry divides the overlapping portion into the road surface imageand the object image on the basis of the road surface color detected bythe road surface color detector.
 7. The vehicle surrounding area imagegeneration device according to claim 1, wherein the image combiningcircuitry performs image combination by alpha blending, and when theobject exists in the overlapping portion, the image combining circuitryuses two pairs of the road surface image and the object image obtainedby the image dividing circuitry, and sets an α value for the objectimage to 1 and sets an α value for the road surface image to 0 for anarea that is the object image in one of the two pairs and is the roadsurface image in the other of the two pairs.
 8. The vehicle surroundingarea image generation device according to claim 1, wherein the thresholdused in the object distance determining circuitry is set on the basis ofa distance from an end portion of the vehicle in the overhead image to ascreen edge of a display unit on which the overhead image is displayed.9. A vehicle surrounding area display system comprising: a plurality ofimaging devices, mounted on a vehicle, for taking an image of a vehiclesurrounding area; a sensor for detecting an object in the vehiclesurrounding area; a vehicle surrounding area image generation device forgenerating an overhead image including the vehicle and the vehiclesurrounding area, on the basis of a plurality of images taken by theimaging devices; and a display for displaying the overhead imagegenerated by the vehicle surrounding area image generation device,wherein the vehicle surrounding area image generation device includes anobject distance determining circuitry to determine whether a distancebetween the object and the vehicle is within a threshold, on the basisof object information acquired by the sensor, an image convertingcircuitry to convert each of the plurality of images into an image foran overhead image, an image dividing circuitry to divide the overlappingportion in the image for an overhead image into a road surface image andan object image when the object exists in an overlapping portion ofimaging areas of the plurality of imaging devices, and an imagecombining circuitry to combine all or part of the image for an overheadimage obtained by the image converting circuitry and the road surfaceimage and the object image obtained by the image dividing circuitryrespectively, to generate the overhead image.
 10. The vehiclesurrounding area display system according to claim 9, further comprisinga vehicle traveling information detector to acquire vehicle travelinginformation including a traveling direction and a speed of the vehicle,from the vehicle.
 11. The vehicle surrounding area display systemaccording to claim 9, wherein the sensor is a millimeter wave radar, asonar, or a camera, and a detection range by the sensor includes theoverlapping portion.
 12. A vehicle surrounding area display method forgenerating and displaying an overhead image including a vehicle and avehicle surrounding area, the vehicle surrounding area display methodcomprising: acquiring image information of a plurality of taken imagesof the vehicle surrounding area, object information including positioninformation of an object in the vehicle surrounding area, and vehicletraveling information including a traveling direction and a speed of thevehicle; converting each of the acquired plurality of images into animage for an overhead image; detecting a road surface color of theoverlapping portion when the object exists in an overlapping portion ofthe plurality of images and a distance between the object and thevehicle is within a threshold; dividing the overlapping portion in theimage for an overhead image into a road surface image and an objectimage on the basis of the detected road surface color; combining all orpart of the image for an overhead image and the road surface image andthe object image, to generate the overhead image; and displaying thegenerated overhead image.
 13. The vehicle surrounding area displaymethod according to claim 12, further comprising selecting a combiningmethod for the overhead image on the basis of the image information of aplurality of taken images of the vehicle surrounding area, objectinformation including position information of an object in the vehiclesurrounding area, and vehicle traveling information including atraveling direction and a speed of the vehicle.
 14. The vehiclesurrounding area display method according to claim 13, wherein, a firstcombining method in which the image for an overhead image, the roadsurface image, and the object image are used is selected when the objectexists in the overlapping portion, the distance between the object andthe vehicle is within the threshold, and the object exists in thetraveling direction of the vehicle.
 15. The vehicle surrounding areadisplay method according to claim 13, wherein, a second combining methodin which the image for an overhead image is used is selected when theobject does not exist in the overlapping portion, when the distancebetween the object and the vehicle is larger than the threshold, or whenthe object does not exist in the traveling direction of the vehicle. 16.The vehicle surrounding area display method according to claim 12,wherein, image combination is performed by alpha blending, and when theobject exists in the overlapping portion, two pairs of the road surfaceimage and the object image are used, and an α value for the object imageis set to 1 and an α value for the road surface image is set to 0 for anarea that is the object image in one of the two pairs and is the roadsurface image in the other of the two pairs.